Abstract 


The aim of this study was to develop the STEM Pedagogical Content Knowledge Scale 
(STEMPCK Scale). Exploratory and confirmatory factor analyses were conducted to examine 
the structural validity of the scale. Exploratory factor analysis of the scale was conducted using 
data from 443 preservice teachers who were studying to become science teachers, preschool 
teachers, elementary school teachers, and mathematics teachers. The confirmatory factor 
analysis of the scale was conducted using data from 212 students who were enrolled in the same 
departments. The scale was administered to 655 preservice teachers in total. To determine the 
reliability of the scale, the Cronbach’s Alpha internal consistency coefficient, the corrected total 
item correlation, and the significance of the differences between the item averages of the top 
27% and bottom 27% were examined by t-test. According to the results obtained, the 
STEMPCK Scale consists of six factors: 21st-Century Skills, Pedagogical Knowledge, 
Mathematics, Science, Engineering, and Technology. The scale included 56 items. The 
Cronbach’s Alpha values of the factors ranged from 0.878 to 0.90, and the corrected item total 
score correlations ranged from 0.306 to 0.895. The results of the t-test showed that all the 
differences between the mean scores of the top 27% and bottom 27% on the items were 
significant. The results of the analyses indicate that the instrument has reasonable internal 
consistency and that the theoretical structure was supported by empirical data. These results 
indicate that the STEMPCK Scale is an appropriate tool for measuring preservice teachers’ 
STEM pedagogical content knowledge. Implications and suggestions for further studies are 
included. 
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Abstract 


The aim of this study was to develop the STEM Pedagogical Content Knowledge Scale 
(STEMPCK Scale). Exploratory and confirmatory factor analyses were conducted to examine 
the structural validity of the scale. Exploratory factor analysis of the scale was conducted 
using data from 443 preservice teachers who were studying to become science teachers, 
preschool teachers, elementary school teachers, and mathematics teachers. The confirmatory 
factor analysis of the scale was conducted using data from 212 students who were enrolled in 
the same departments. The scale was administered to 655 preservice teachers in total. To 
determine the reliability of the scale, the Cronbach’s Alpha internal consistency coefficient, 
the corrected total item correlation, and the significance of the differences between the item 
averages of the top 27% and bottom 27% were examined by f-test. According to the results 
obtained, the STEMPCK Scale consists of six factors: 21st-Century Skills, Pedagogical 
Knowledge, Mathematics, Science, Engineering, and Technology. The scale included 56 
items. The Cronbach’s Alpha values of the factors ranged from 0.878 to 0.90, and the 
corrected item total score correlations ranged from 0.306 to 0.895. The results of the f-test 
showed that all the differences between the mean scores of the top 27% and bottom 27% on 
the items were significant. The results of the analyses indicate that the instrument has 
reasonable internal consistency and that the theoretical structure was supported by empirical 
data. These results indicate that the STEMPCK Scale is an appropriate tool for measuring 
preservice teachers’ STEM pedagogical content knowledge. Implications and suggestions for 
further studies are included. 
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Scientific and technological progress requires workers to have different skills than in the past. In 
addition to foundational skills like literacy and numeracy, they need competencies such as problem 
solving, critical thinking, creativity, and collaboration and character qualities such as curiosity. To 
teach the skills that meet the needs of a 21st-century marketplace, countries have made reforms in 
their educational systems (Drew, 2011). Science, technology, engineering, and mathematics (STEM) 
lie at the heart of these reform movements. Through the integration of science, technology, 
engineering, and mathematics, students would be able to study the challenges that arise in today’s 
world such as energy shortages, environmental problems, and health problems (Bybee, 2010). 
Students fully equipped with STEM knowledge will be able to identify, apply, and integrate its 
concepts to understand complex problems and generate solutions to solve those 


problems (Meng, Idris, Eu, & Daud, 2013). Successful integration of STEM largely “depends on 
whether teachers develop a solid understanding of subject matter and conceptualize connections 
among subjects” (Pang & Good, 2000, p. 77). Many teachers have holes in their subject content 
knowledge; therefore, asking teachers to teach another subject may create new knowledge gaps, 
challenges, and difficulties (Stinson, Harkness, Meyer, & Stallworth, 2009). 

To successfully implement STEM education in schools, teachers should understand what STEM 
means and should be equipped with STEM-related skills and knowledge of STEM. Teachers should 
apply effective teaching methods and techniques in their classroom to support students’ learning of 
STEM areas (Lichtenberger & George-Jackson, 2013). Furthermore, teachers should acknowledge 
students’ success in STEM-related fields (Beier & Rittmayer, 2008). It is important for teachers to 
have strong content and pedagogical knowledge in STEM (Kennedy, Ahn, & Choi, 2008). 


In the STEM education research literature, there is a no clear agreement about the concept of 
pedagogical content knowledge (PCK) and content knowledge for teaching. In spite of this lack 
of agreement, researchers use Shulman’s model as a basis for research about teachers’ PCK. 
According to Shulman (1986), several domains of knowledge influence how teachers teach. 
These domains are: content knowledge, general pedagogical knowledge (i.e., knowledge of 
strategies for classroom management), curriculum knowledge, PCK, knowledge of learners and 
their characteristics, knowledge of context (i.e., classroom), and knowledge the purposes and 
values of education (Cochran, DeRuiter, & King, 1993; Hume & Berry, 2011; Shulman, 1986). 
During the years, other researchers (Appleton, 2003; Cochran et al., 1993; Magnusson, Krajcik, 
& Borko, 1999) have built on or challenged Shulman’s approach by suggesting different views of 
PCK. From another point of view, PCK has its own unique identifiers (Magnusson et al., 1999). 
Based on these models, PCK is a transformation of knowledge from other knowledge categories 
(e.g., orientations toward science teaching, knowledge and beliefs about science curriculum, 
knowledge and beliefs about students’ understanding of specific science topics, knowledge and 
beliefs about assessment in science, and knowledge and beliefs about instructional strategies for 
teaching science; Magnusson et al., 1999). 

Numerous methods of measuring teachers’ PCK have been developed since Shulman introduced 
it (Baxter & Lederman, 1999): the lesson planning method (developed by van der Valk and 
Broekman, 1999), the use of metaphors, the use of “classroom window” cases (developed by 
Loughran, Milroy, Berry, Gunstone, and Mulhall, 2001), expert—-novice studies, interviews, 
classroom observations, and teacher focus groups (Abell, 2007; Baxter & Lederman, 1999; Hume 
& Berry, 2011; Loughran, Berry, & Mulhall, 2012). All these techniques are divided into three 
categories: (1) convergent and inferential techniques; (2) concept mapping, card sorts, and 
pictorial representations; and (3) multimethod evaluations (Baxter & Lederman, 1999). 

In a review of research published in science education, different tools were used for gathering 
information about teacher practices, content knowledge, and PCK in STEM education interventions 
(Minner & Martinez, 2013; Minner, Martinez, & Freeman, 2013). These instruments are observation 
protocols such as the Instructional Strategies Classroom Observation Protocol (ISCOP), the Lesson 
Flow Classroom Observation Protocol (LFCPO), the Science Teacher Inquiry Rubric (STIR), 
Assessment of the Facilitation of Mathematizing (AFM), and Early Mathematics Classroom 
Observation (EMCO); scoring rubrics such as Transforming Instruction by Design in Earth Science 
(TIDES) and Scoop Notebook (Scoop); surveys such as the Indiana Mathematics Beliefs Scale 
(IMBS) and the Science Teaching Efficacy Belief Instrument (STEBI); protocols such 


as Danielson’s Framework for Teaching Domains (FFT); and interviews such as Views of 
Nature of Science Form C (VNOS-C; Minner & Martinez, 2013; Minner et al., 2013). Most 
instruments have been developed to assess students’ knowledge and then adapted for teachers 
(Minner & Martinez, 2013). Furthermore, it seems that current instruments are limited to 
assessing teachers’ PCK in STEM (STEMPCK) and that more effort needs to be made to 
develop new instruments for STEM educational interventions (Benuzzi, 2015; Hudson, English, 
Dawes, King, & Baker, 2015; Rogers, Winship, & Sun, 2016; Stohlmann, Moore, & Roehrig, 
2012). This study will hopefully begin to fill that gap by not only describing STEMPCK but also 
explaining how these areas will be assessed through one instrument. 

To effectively integrate STEM education, teachers should adequately build STEMPCK. 
STEMPCK is a combination of different categories: STEM content knowledge, STEM integration 
knowledge, pedagogical knowledge, 2Ist-century skills knowledge, and context knowledge 
(Appleton, 1995, 2003; Avery, 2009; Ball, Thames, & Phelps, 2008; Benuzzi, 2015; Custer & 
Daugherty, 2009; Eckman, Williams, & Silver-Thorn, 2016; Eilks & Markic, 2011; Epstein & Miller, 
2011; Harris & Hofer, 2011; Hill, Rowan, & Ball, 2005; Hudson et al., 2015; C. A. Lee 
& Houseal, 2003; K.-T. Lee & Nason, 2012; Nadelson, Seifert, Moll, & Coats, 2012; National 
Research Council, 2000; President’s Council of Advisors on Science and Technology [PCAST], 
2010; Rogers et al., 2016; Stohlmann et al., 2012; Shulman, 1986; Wilson, 2011; Wang, Moore, 
Roehrig, & Park, 2011). 


Content knowledge is defined as “the amount and organization of knowledge per se in the 
mind of the teacher” (Shulman, 1986, p. 9). This knowledge includes different components 
including knowledge of concepts (including principles and definitions), theories, ideas, 
organizational frameworks, knowledge of evidence and proof, and the knowledge construction 
process (Shulman, 1986). Teachers who have strong knowledge and skills in the STEM 
disciplines can easily integrate STEM education into practice and develop effective STEM 
teaching strategies (Abell, 2007; Ball et al., 2008; Bukova-Giizel, Cantiirk-Giinhan, Kula, Ozeiir, 
& Elc¢i, 2013; Nadelson et al., 2012; PCAST, 2010). Teachers’ effectiveness in teaching STEM 
subjects is related to their competence in the individual subjects (Eckman et al., 2016; C. A. Lee 
& Houseal, 2003). Teachers of STEM disciplines must have a conceptual understanding of how 
the STEM subjects are interactive and adaptable (Ostler, 2012). Besides having a broad 
knowledge base of STEM content, teachers must witness advanced integrated pedagogical 
models to be able to integrate these practices into their classroom teaching (Ostler, 2012). 
Furthermore, STEM integration knowledge is also related to STEMPCK and PCK. Teachers 
with strong STEMPCK do not avoid teaching STEM subjects and integrating STEM disciplines 
in the classroom (Yildirim, 2016; Wang et al., 2011). 


Pedagogical knowledge refers to knowledge of teaching methods, knowledge of classroom 
assessment, and knowledge of learning processes (Shulman, 1986). Teachers with deep 
pedagogical knowledge understand how students construct knowledge; understand cognitive, 
social, and developmental theories of learning; understand how such theories apply to students in 
their classroom; and are aware of what is going on in all parts of the classroom and are able to 
handle several classroom events. In short, pedagogical knowledge includes all issues related to 
student learning, classroom management, knowledge of assessments, knowledge of student 
characteristics, and lesson plan development and implementation (Yusof, Zakaria, & Maat, 2012; 
Briscoe & Peters, 1997; Shulman, 1986). 


Twenty-first-century skills knowledge describes the skills and knowledge that all students 
must master to succeed in life and work. Twenty-first-century skills should be taught in 
conjunction with core academic knowledge (such as music, mathematics, and science) and other 
essential skills, such as learning and innovation skills (including critical thinking and problem 
solving, creativity and innovation, communication and collaboration, visual literacy, scientific 
and numerical literacy, cross-disciplinary thinking, and basic literacy); information, media, and 
technology skills (including information literacy; media literacy; and _ information, 
communications, and technology literacy); and life and career skills (including flexibility and 
adaptability, initiative and self-direction, social and cross-cultural skills, productivity and 
accountability, and leadership and responsibility; Binkley et al., 2012; Voogt & Roblin, 2010). 

Context knowledge refers to knowledge of internal and external resources. Internal sources 
include reflection on personal experiences of teaching, and external sources include subject- 
matter knowledge and government rules about education and policies (Barnett & Hodson, 2001). 
Context defines the environment for teaching and learning that includes content. A classroom, 
the characteristics of the region, the culture of the school, students’ backgrounds, the province of 
the school, and the region of the school also affect the effectiveness and quality of STEM 
education (Harris & Hofer, 2011; Shulman, 1986; Yusof et al., 2012). 

Up to this point, we have summarized the research on teachers’ PCK and proposed a new model, 
the STEMPCK model, to teach and train in-service and preservice teachers. Based on an intensive 
literature review and interviews with preservice teachers and engineering students, we hypothesized 
that STEMPCK has different categories: content knowledge, integration knowledge, pedagogical 
knowledge, 21st-century skills knowledge, and context knowledge. Although there have been many 
studies conducted to evaluate STEM teachers’ beliefs and to develop teachers’ experiences in the 
classroom while integrating STEM curricula, there is no single instrument to evaluate teachers’ 
STEMPCK. Because it is difficult and expensive to score open-ended questions, concept maps, and 
comments on videotaped lessons, there is a need for new instruments to evaluate teachers’ 
STEMPCK. Therefore, in this study, we developed a paper-and-pencil test: the STEM Pedagogical 
Content Knowledge Scale (STEMPCK Scale) for preservice teachers. We believe that this study 
makes an important contribution to research on STEMPCK. 


Methods 


In this study, an instrument, the STEMPCK Scale, was developed to determine preservice 
teachers’ STEMPCK. 


Participants 


The study was conducted during the fall semester of 2016 in a public university located in the 
eastern part of Turkey. The participants were preservice teachers from different departments, 
including Science Education, Mathematics Education, Early Childhood Education, and 
Elementary School Education. 


Research Group 


Two different groups of preservice teachers participated in this study. The first group consisted of 


443 preservice teachers who were studying in the Science Education, Mathematics Education, 


Early Childhood Education, and Elementary School Education departments. The second group 
consisted of 212 preservice teachers who were studying in the same departments. The 
exploratory factor analysis of the developed scale was conducted using data from the first group, 
and the confirmatory factor analysis was conducted using data from the second group. Because 
repeating the study and either redoing the exploratory factor analysis or conducting a 
confirmatory factor analysis after an exploratory factor analysis in the first sample or conducting 
two successive confirmatory factor analyses would not validate the supposed structure, different 
samples were used in the study. Frequency and percentage distributions of the preservice 
teachers in the first and second groups by their departments are shown in Table 1. 

Table 1 shows that 11.29% of the participants in Study Group | were preservice science 
teachers, 18.06% were preservice mathematics teachers, 27.99% were preservice early childhood 
teachers, and 42.95% were preservice elementary school teachers. In Study Group 2, 16.51% of 
the participants were preservice science teachers, 19.81% were preservice mathematics teachers, 
25.94% were preservice early childhood teachers, and 37.74% were preservice elementary 
school teachers. The distribution of preservice teachers by grade level is shown in Table 2. Table 
2 shows that 33.74% of the preservice teachers were sophomore students, 35.73% were juniors, 
and 30.53% were seniors. 

A sample consisting of 655 preservice teachers was selected for the exploratory and confirmatory 
factor analyses of the STEMPCK Scale. Exploratory and confirmatory factor analyses of the scale 
were performed using this sample. According to Comrey and Lee (1992), because this scale’s sample 
group is 655, it has a “very good” rating (p. 217). Additionally, Tavsancil (2002) also points out that 
to carry out factor analysis for a scale, the sample should be five to 10 times larger than the number 
of items. In this study, the sample is five times larger than the number of items. 


Development of the STEMPCK Scale 


In the process of developing an instrument to measure teachers’ STEMPCK, five stages were 


involved. 
Table 1 
Frequency and Percentage Distribution of Preservice Teachers in Study Group by Department and Major 
Study Group 1 Study Group 2 
Department and major f Jo f Jo 
Department of Mathematics and Science Education 
Science Education 50 11.29 35 16.51 
Mathematics Education 80 18.06 42 19.81 
Department of Primary Education 
Early Childhood Education 124 27.99 55 25.94 
Elementary School Education 189 42.66 80 37.74 


Total 443 100 212 100 


Table 2 
Frequency and Percentage Distribution of Preservice Teachers by Grade Level (N = 655) 


Major f Jo 
Sophomore 
Science Education 47 7.18 
Mathematics Education 48 7.33 
Early Childhood Education 55 8.40 
Elementary School Education 71 10.84 
Total 221 33.74 
Junior 
Science Education 21 3.21 
Mathematics Education 40 6.11 
Early Childhood Education 60 9.16 
Elementary School Education 113 17.25 
Total 234 35.73 
Senior 
Science Education 17 2.60 
Mathematics Education 37 5.65 
Early Childhood Education 61 9.31 
Elementary School Education 85 12.98 
Total 200 30.53 


Stage 1. To develop the STEMPCK Scale’s items, interviews were conducted with 40 
students from different education majors, including Science Education, Mathematics Education, 
Early Childhood Education, and Elementary School Education, and 17 students from the 
Computer Science department. In the interviews, students were asked open-ended questions 
regarding what they thought about content knowledge, STEM content knowledge, PCK, 21st- 
century skills, STEM integration knowledge, and STEM integration. 

Stage 2. Before the scale was developed, an extensive review of the literature was conducted. 
This literature review included studies focused on the STEM disciplines; teaching the STEM 
disciplines; and scales to determine teachers’ PCK, technological PCK, and engineering PCK (e.g., 
Aksu, Metin, & Konyaloglu, 2014; Bukova-Giizel et al., 2013; Brenneman, 2011; Campbell, Abd- 
Hamid, & Chapman, 2010; Enochs, Smith, & Huinker, 2000; Faber et al., 2013; Halim, Mohd 
Meerah, Zakaria, Syed Abdullah, & Tambychik, 2012; Kelleys & Knowles, 2016; Kiray, 2016; 
Kloosterman & Stage, 1992; Koehler et al., 2011; Onal, 2016; Ryang, 2014; Schmidt et al., 2009; 
Unfried, Faber, Stanhope, & Wiebe, 2015; Viiri, 2003; Yildirim & Selvi, 2015, Yusof et al., 2012). 

Stage 3. The knowledge gained from the literature and the data gathered from interviews 


were used to develop a draft scale that included 72 items about STEMPCK. These items were 
placed under the themes: STEM content knowledge, STEM integration knowledge, pedagogical 
knowledge, 21st-century skills knowledge, and context knowledge. 

Stage 4. Afterwards, to ensure content validity of the STEMPCK Scale, the draft scale was 
sent to six experts. Four of those experts were in the STEM field; one was in the Evaluation, 
Measurement, and Research Design department; and the last was an expert in PCK. The experts 
were asked to examine items regarding to their relevance to content coverage, understandability, 
and consistency among items. Experts’ suggestions about clarity of items were taken into 
consideration, and any necessary changes were made. 

Stage 5. The final version of the 72-item draft scale was administered to 60 preservice 
teachers. Preservice teachers checked the comprehensibility of all developed items. Then, the 
items on the scale were rearranged based on statistical results. The last version, consisting of 56 
items that were rated on a 5-point scale, was administered to 655 preservice teachers. Preservice 
teachers’ responses were analyzed using SPSS Statistics (Version 17) and LISREL (Version 8). 


Factor Analysis of the STEMPCK Scale and Reliability Analyses 


To conduct a descriptive factor analysis of the scale, the final draft of the STEMPCK instrument 
was administered to 433 preservice teachers. The confirmatory factor analysis of the scale was 
performed using data from 212 preservice teachers. The validity and reliability analyses of the scale 
were calculated using SPSS Statistics (Version 17) and LISREL (Version 8). The results of the 
exploratory and confirmatory factor analyses of the STEMPCK Scale are explained below. 

Results of the exploratory factor analysis. Factor analysis is a collection of methods used to 
examine how underlying factors influence the variable (Biiyiik6ztiirk, 2006). There are two types of 
factor analyses: exploratory factor analysis and confirmatory factor analysis. Exploratory factor 
analysis attempts to investigate underlying factors, and confirmatory factor analysis tests how well 
the measured variables represent the number of constructs. Exploratory factor analysis was used in 
this study to determine the factoring situation of the items in the scale and their factor loads. 

After conducting the exploratory factor analysis, the researchers performed a confirmatory factor 
analysis on the scale. The scale’s compliance with the factor analysis was assessed with the Kaiser- 
Meyer-Olkin (KMO) test and the Bartlett test. The KMO value of the 56-item scale was calculated as 
0.88, and the Bartlett test was determined to be significant (v2 = 7722,135, df= 1540, p < .05). The 
data were determined to be suitable for factor analysis because the KMO coefficient was bigger than 
0.60 and because the Bartlett test was significant (Bitytik6ztiirk, 2006). 

Varimax analysis was applied to bring together factors with high correlations (Dogan, 2011). 
Based on the varimax analysis, three factors’ eigenvalues were found to be greater than 1. To 
calculate eigenvalue, a scree plot method was used. A scree plot shows the eigenvalues on the y- 
axis and the number of factors on the x-axis. Figure 1 shows the maximum number of factors. 
Additionally, after the sixth factor, the graph shows accelerated decline, meaning that seventh 
and eight factors’ variances were close to each other. This shows that the scale has a six-factor 
structure (Bityiik6ztiirk, 2006). 

As shown in Figure 1, the eigenvalues of the six factors on the scale (21st-Century Skills 
Knowledge, Pedagogical Knowledge, Mathematical Knowledge, Science Knowledge, Engineering 
Knowledge, and Technology Knowledge) were 14.928, 4.354, 3.065, 2.315, 2.267, and 2.00, 


Eigenvalue 


Factor Number 
Figure 1. Scatter graph. 


respectively. These values were above 1. However, the percentages of variance explained by the 
factors were 26.657%, 7.774%, 5.774%, 4.133%, 4.049%, and 3.571%, respectively. 


Reliability analysis. The STEMPCK Scale’s internal consistency coefficient was calculated, 
and the Cronbach’s Alpha values are presented in Table 3. The Cronbach’s Alpha value was .95 
for the entire STEMPCK Scale, .90 for the 21st-Century Skills Knowledge dimension, .87 for the 
Pedagogical Knowledge dimension, .85 for the Mathematical Knowledge dimension, .86 for the 
Science Knowledge dimension, .81 for the Engineering Knowledge dimension, and .78 for the 
Technology Knowledge dimension. 

The variance quantities were ranked as follows: 21st-Century Skills Knowledge was 26.657%, 
Pedagogical Knowledge was 7.774%, Mathematical Knowledge was 5.774%, Science Knowledge 
was 4.133%, Engineering Knowledge was 4.049%, and Technology Knowledge was 
3.571% (see Table 4). After factor rotation, the number of items for each factor was determined: 
21st-Century Skills Knowledge included 15 items with factor loads ranging from 0.423 to 0.760, 
Pedagogical Knowledge consisted of 12 items with factor loads ranging from 0.434 to 0.724, 
Mathematical Knowledge consisted of eight items with factor loads ranging from 0.435 to 0.811, 
Science Knowledge consisted of eight items with factor loads ranging from 0.524 to 0.785, 
Engineering Knowledge consisted of seven items with factor loads ranging from 0.493 to 0.754, and 
Technology Knowledge consisted of seven items with factor loads ranging from 0.463 to 0.734. The 
results indicate that the SPAB’s items corrected item-total correlation ranged from 0.306 to 


Table 3 
Cronbach’s Alpha Values for the STEMPCK Scale and Its Components 
Dimension Cronbach’s o 
STEMPCK Scale 95 
21st-Century Skills Knowledge .90 
Pedagogical Knowledge 87 
Mathematical Knowledge 85 
Science Knowledge .86 
Engineering Knowledge 81 


Technology Knowledge 78 


0.895. T-tests comparing the total scores of the bottom 27% and top 27% for each item indicated 
that there was an important difference in scores for all items. The scores of the top 27% and 
bottom 27% of the total group tell us whether an item has discriminated the high scorers and low 
scorers on the scale. 

As shown in Table 5, the correlations between the 21st-Century Skills Knowledge and 
Pedagogical Knowledge factors (r = .425), the 21st-Century Skills Knowledge and Mathematical 
Knowledge factors (r = .417), the 21st-Century Skills Knowledge and Science Knowledge factors 


Table 4 
Results of the Exploratory Factor Analysis of the STEMPCK Scale 
Item-total t-value Rotator factor Common 
Item M SD correlation (bottom 27%, top 27%). load factor load 


21st-Century Skills Knowledge 


CS4 4.42 714 S11 1.275 .760 .626 
CS5 4.26 726 534 1.302 £107 599 
CS7 4.40 194 489 0.665 695 571 
CS3 4.34 718 536 2.101 691 583 
CS1 4.30 718 74 1.275 .607 542 
CS9 4.45 915 384 0.601 595 416 
CS13 4.26 718 595 0.400 581 S11 
CS11 4.16 856 438 0.988 563 393 
CS15 4.42 795 449 0.957 562 427 
CS12 4.09 791 535 1.642 562 448 
CS2 4.21 .700 563 1.801 539 487 
CS6 3.99 843 558 0.906 526 425 
CS8 4.14 814 540 0.800 447 381 
CS14 4.07 857 402 2.359 423 .436 
Pedagogical Knowledge 
PK12 4.13 107 508 1.311 724 584 
PKS 3.96 750 480 1.127 .689 541 
PK10 4.15 719 403 1.847 .674 488 
PK11 3.79 825 445 0.400 .633 476 
PK13 4.17 726 465 1.839 629 501 
PK6 4.16 722 487 2.532 613 472 
PK4 4.16 .662 493 0.406 .601 452 
PK1 4.33 817 405 0.781 559 .400 
PK7 4.33 .676 539 1.948 543 534 
PK8 4.25 703 467 0.994 528 451 
PK9 4.04 117 430 2.052 518 361 


PK3 3.85 861 435 1.371 434 312 


Mathematical Knowledge 


M2 3.91 .908 514 1.268 811 TAS 
M4 3.49 = 1.05 433 0.781 .706 542 
M6 3.74 .957 440 0.986 .698 .608 
M1 3.68  .969 492 1.602 .698 75 
M9 3.39 = .950 439 0.566 .668 530 
M3 3.15 .858 341 0.307 .662 .600 
M5 3.82  .870 347 1.766 495 307 
M10 3.52 = 1.08 490 1.426 .405 405 


Science Knowledge 


$5 3.34 =.902 495 2.613 785 .682 
$7 2.85 1.04 .498 0.713 .710 596 
S4 3.50 .916 502 1.205 .703 611 
S9 3.48 = 1.21 369 0.783 .682 529 
83 3.61  .909 442 1.171 .673 563 
Sl 3.67 890 396 1.443 .635 75 
S8 3.55 .960 451 1.128 547 460 
82 3.04 = .980 .408 0.863 524 417 


Engineering Knowledge 


E4 2.95 1.11 341 2.827 .754 .616 
E5 3.34 1.16 353 1.675 .695 593 
E2 2.68 .964 346 2.377 .689 541 
E3 2.51 .976 341 1.977 .682 75 
E7 2.82  .996 317 2.530 .657 482 
E9 3.90  .990 372 0.692 502 363 
E8 4.00 941 323 0.818 .493 327 
Technology Knowledge 
T6 3.29 .977 401 0.430 .734 625 
Tl 3.43 837 322 0.992 .692 516 
T4 3.66 .933 .410 0.434 .649 504 
TS 3.37 = 922 .462 1.130 .616 497 
T9 3.37 = .937 306 1.507 511 368 
T2 4.02 .741 .450 0.854 509 451 
T10 4.00  .699 426 0.484 .463 331 


Note. Factor load values lower than .30 are not shown in this table (Cokluk, Sekercioglu, & Biiyiikéztirk, 
2014). 

a The results of the t-value comparing the 27% of the students and the top and the 27% at the bottom 
shows that there is a significant difference in scores from all items (Wiersma & Jurs, 1990). 


(r = .421), the 21st-Century Skills Knowledge and Engineering Knowledge factors (r = .261), and 
the 21st-Century Skills Knowledge and Technology Knowledge factors (r = .660) were statistically 
significant (p < .01). Moreover, the correlations between the Pedagogical Knowledge and 
Mathematical Knowledge factors (r = .389), the Pedagogical Knowledge and Science Knowledge 
factors (r = .361), the Pedagogical Knowledge and Engineering Knowledge factors (r = .508), and 
the Pedagogical Knowledge and Technology Knowledge factors (r = .427) were significant (p < 
.01). In addition, the correlations between the Mathematical Knowledge and Science Knowledge 
factors (r = .485), the Mathematical Knowledge and Engineering Knowledge factors (r = .323), 
and the Mathematical Knowledge and Technology Knowledge factors (r = .474) were significant 
(p < .01). Similarly, the correlations between the Science Knowledge and Engineering Knowledge 
factors (r = .330, p < .01) and the Science Knowledge and Technology Knowledge factors (r = 
.507) were significant (p < .01). In brief, the correlations between the subdimensions of the 
STEMPCK Scale appeared to be significantly positive. The lowest correlation between the 
subdimensions of the STEMPCK Scale was 0.261, and the highest correlation was 0.660. 

Results of the confirmatory factor analyses. As mentioned previously, exploratory factor 
analysis of the STEMPCK Scale was conducted with Study Group 1, and confirmatory factor 
analysis was conducted with Study Group 2. Study Group 2 consisted of 212 preservice teachers 
in the Mathematics Education, Science Education, Early Childhood Education, and Elementary 
School Education programs. A confirmatory factor analysis using the structural equation model 
was conducted to determine the existing structure of the scale (see Figure 2). 

Model conformity of the STEMPCK Scale was tested by criteria such as the Goodness of Fit 
Index (GFD, the Adjusted Goodness of Fit Index (AGFI), the Root Mean Square Error of 
Approximation (RMSEA), the Comparative Fit Index (CFD, and the Normed Fit Index (NFI; Kilig & 
Sen, 2014). Chi-square (2), ¥2/SD, RMSEA, RMR, GFI, and AGFI are commonly used in 
confirmatory factor analysis. In large samples, calculated y2/df ratio can also be used as a criterion 


Table 5 
Correlation Values for STEMPCK Scale Components’ Mean Values and Standard Deviations 


21st-Century 
Skills Pedagogical Mathematical Science Engineering Technology 


Factor Knowledge Knowledge Knowledge Knowledge Knowledge Knowledge 
21st-Century --- 
Skills Knowledge 
Pedagogical A25** --- 
Knowledge 
Mathematical ALT** 389%* a3 
Knowledge 
Science 421** 361 ** A85** --- 
Knowledge 
Engineering .261** 508** 323** 330** aes 
Knowledge 
Technology .660** AQT * AT4** 07 ** .280** --- 
Knowledge 
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Chi-Square = 1701.54 df= 1469 p-value = .00002 RMSEA = 0.034 


Figure 2. Confirmatory factor analysis of the STEMPCK Scale. 


Table 6 
Fit Indices of STEM Pedagogical Content Knowledge and Acceptable Fit Indices Values 


Chi-square p-value CFI NFI GFI AGFI IFI SRMR RMSEA 
value 
1701.54 p< .05 0.93 0.93 0.99 0.97 0.93 0.043 0.034 


for conformity adequacy. If calculated y2/df ratio is (1) y2 / df < 3, it can be regarded as good fit; 
if it is up to (2) y2 / df <S, it can be regarded as sufficient. 


If the GFI and AGFI values are higher than 0.90 (Hooper, Coughlan, & Mullen, 2008; 
Siimer, 2000) and the RMR and RMSEA values are lower than 0.05 (Joreskog & Sdrbom, 1993; 
Siimer, 2000), model—data fit is good. However, if the GFI value is higher than 0.85, the AGFI is 
higher than 0.80, and the RMR and RMSEA values are lower than 0.080, model—data fit is 
acceptable (Anderson & Gerbing, 1984; Hu & Bentler, 1999; Siimer, 2000). The results of the 
confirmatory factor analysis for the STEMPCK Scale are shown in Table 6. 


According to the confirmatory factor analysis results, conformity between structural equation 
model and scale was high. Additionally, the Chi-square value was found to be significant. The 
value of y2 depends on the size of the sample, and when the size of the sample increases, it 
provides significant results. Briefly, when the Chi-square value (v2) is divided by the value of 
the degrees of freedom (df), it shows that the value is less than 3; in other words, based on the 
results, the model—data fit was high. In addition, if the CFI, NFI, and AGFI values are above 
0.90, it indicates that the model—data fit is good. Also, if RMSEA value is 0.034, it indicates that 
the model—data fit is high. According to confirmatory factor analysis, it was determined that the 
STEMPCK Scale consists of six subdimensions and that model—data fit was high. 


Discussion 


The STEM disciplines (science, technology, engineering, and mathematics), Shulman’s 
model of PCK, and theoretical knowledge were used as a basis for the STEMPCK Scale, and 
items were written based on extensive review of the literature (e.g., Aksu et al., 2014; Bukova- 
Giizel et al., 2013; Brenneman, 2011; Campbell et al., 2010; Enochs et al., 2000; Faber et al., 
2013; Kelleys & Knowles, 2016; Kiray, 2016; Kloosterman & Stage, 1992; Koehler et al., 2011; 
Onal, 2016; Ryang, 2014; Schmidt et al., 2009; Unfried et al., 2015; Viiri, 2003; Yusof et al., 
2012) and interviews. The results of the varimax analyses shaped the factors in the STEMPCK 
Scale. The STEMPCK Scale consisted of six factors: 21st-Century Skills Knowledge, 
Pedagogical Knowledge, Mathematical Knowledge, Science Knowledge, Technology Knowledge, 
and Engineering Knowledge. The total variance of these six factors was 51.958%, and the 
Cronbach’s Alpha value of the scale was 0.95. The Cronbach’s Alpha value was calculated as 
0.90 for 21st-Century Skills Knowledge, 0.87 for Pedagogical Knowledge, 0.85 for Mathematical 
Knowledge, 0.86 for Science Knowledge, 0.81 for Engineering Knowledge, and 0.78 for 
Technology Knowledge. These results show that the developed STEMPCK Scale is reliable. 


Additionally, based on the confirmatory factor analysis results, the CFI, GFI, IFI, NFI, and 
AGFI values were higher than 0.90, indicating that model—data fit was high (Hooper et al., 2008; 
Siimer, 2000). In addition, the SRMR and RMSEA values (0.043 and 0.034, respectively) were 


less than 0.05, indicating that model—data fit was high (Anderson & Gerbing, 1984; Hooper et 
al., 2008; Jéreskog & Sérbom, 1993; Siimer, 2000; Hu & Bentler, 1999). According to the 
confirmatory factor analysis results, model—data fit was high, and the STEMPCK Scale was 
found to have six subdimensions. This scale was found to be valid and reliable based on the 
results of the exploratory and confirmatory factor analyses. 


The STEMPCK Scale was developed to assess preservice teachers’ STEMPCK. 
Understanding of teacher candidates’ STEMPCK will help teacher educators strengthen their 
courses and identify gaps in teacher candidates’ knowledge that need to be addressed. 


Conclusion 


The STEMPCK Scale is a valid and reliable instrument that will be used to measure 
STEMPCK for prospective teachers. A review of the literature found that there were no valid and 
reliable STEMPCK instruments for preservice and in-service teachers. Therefore, this study will 
contribute to this side of the STEMPCK literature. The validity and reliability of the instrument 
was determined using data from 655 students in Science Education, Mathematics Education, 
Elementary School Education, and Early Childhood Education. However, to use this instrument 
with different groups, the validity and reliability of the instrument would first need to be tested 
with those groups. 


Some limitations were present in the study. This study was conducted with preservice 
teachers in the Mathematics Education, Science Education, Early Childhood Education, and 
Elementary School Education programs in the fall semester of 2017 in a public university in 
Turkey. Additional research is needed to administer the STEMPCK Scale in different groups or 
departments (e.g., the chemistry teaching department or the physics teaching department). 
Furthermore, during the pilot study, interviews were conducted with 40 students from the school 
of education and 17 students from the school of engineering. Because only one department (the 
computer science department) actively accepts students,: only students from that department 
were invited to participate in this study, and the researchers conducted interviews with them. 
Researchers first started the study by reviewing and analyzing the literature about PCK. From 
various different conceptions given by the scholars, the researchers synthesized their own 
conception about the constructs to represent the scope of PCK needed by teachers in order to 
teach STEM subjects in the classrooms. Finally, this is the first study examining this instrument, 
and additional studies are needed to furnish more evidence of construct validity. 


The STEMPCK Scale is a robust instrument that elementary, middle, and high school STEM 
education program leaders and teacher educators can use to identify what preservice and in- 
service teachers know in general and what they do not know. Because this instrument is new, 
researchers are also encouraged to continue testing and refining the scale’s content. Finally, this 
instrument could contribute to studies on teacher professional development and teacher education 
programs. Gaps in teachers’ knowledge that are identified can be used as guidelines for planning 
more effective professional development programs for teachers. 


' Due to an insufficient number of faculty members, our university has not completed the conditions 
required by the Council of Higher Education to accept student to other STEM departments. 
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